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DETAILED ACTION 
Response to Arguments 

1. Applicant's arguments filed 1/18/2011 have been fully considered but they are not persuasive. 

2. Re claims 1,16, and 21 , the Applicant contends that the prior art cited fails to teach or suggest a 
processor configured to "generate, from the buffered wide angle image data, panoramic view data of a 
panoramic view of the monitored area, generate, from the buffered wide-angle image data, virtual view 
data representing a virtual view of a portion of the panoramic view, and encode the panoramic view data 
and the virtual view data for display." However, the Examiner respectfully disagrees. Koyanagi discloses 
that to generate a panoramic picture, the pan filter camera disposed at the center of a virtual spherical 
surface photographs adjacent pictures, which are combined in a way such that distortions from 
overlapping edges are normalized to eliminate distortion (Koyanagi: Fig. 4A-4F and paragraphs [0053]- 
[0054]). Kuban discloses a system which examines a hemispherical distorted image (analogous to the 
non-normalized combined image of Koyanagi) to create smaller, localized window of interest out of the 
overall panoramic image (Kuban: column 5, lines 6-32). Furthermore, the localized window of interest 
has the capability to be controlled by a joystick in order to emulate traditional pan and tilt functions 
(Kuban: column 5, lines 33-39) and to modify the magnification of the localized window similar to a zoom 
function (Kuban: column 5, lines 45-49), wherein the transformations occur between the input memory 
buffer and the output memory buffer (Kuban: column 5, lines 6-10). Therefore, the functions are 
performed with only stored data, and do not utilize a separate pan/tilter to generate the localized window. 
Kuban states that the "components function as a system to select a portion of the input image (fisheye or 
other wide angle) and the mathematically transform the image to provide the proper prospective for 
output" (Kuban: column 4, lines 61-65, emphasis added), wherein the stitched-together panoramic image 
of Koyanagi qualifies as an "other", non-fisheye wide angle image. 

3. Since the Applicant's arguments have been deemed non-persuasive and no claim amendments 
have been presented, the previous rejection is maintained and this action is made final. 
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Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-4, 11-12, 14, 16-18, 20-21, 23 and 26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Koyanagi et al. (US 20040257436 A1) in view of Kuban et al. (US 5359363 A). 

Re claim 1 , Koyanagi discloses a system for creating signals indicative of a graphical user 
interface from wide-angle image data corresponding to a monitored area, said system comprising: a 
buffer configured to receive wide-angle image data corresponding to the monitored area (Koyanagi: Fig. 
3, Storing portion 30); and a processor (Koyanagi: Fig. 3, computer 1 and controlling portion 31) operably 
coupled to said buffer and configured to: generate, from the buffered wide angle image data, panoramic 
view data of a panoramic view of the monitored area (Koyanagi: Fig. 3 and paragraph [0046], computer 1 
comprises a controlling portion 31 , a video capture portion 29, and a storing portion 30; Figs. 1 and 2 and 
paragraph [0051], computer 1 processes graphics in the operation area 6A (corresponding to the claimed 
virtual view) and the panorama operation area 6B (corresponding to the claimed panoramic view) 
displayed on the monitor, and, therefore, the controlling portion 31 of computer 1 must control the 
processing algorithms for generating the graphics in the operation area 6A and the panorama operation 
area 6B); generate virtual view data representing a virtual view of a portion of the panoramic view 
(Koyanagi: Fig. 3 and paragraph [0046], computer 1 comprises a controlling portion 31 , a video capture 
portion 29, and a storing portion 30; Figs. 1 and 2 and paragraph [0051], computer 1 processes graphics 
in the operation area 6A (corresponding to the claimed virtual view) and the panorama operation area 6B 
(corresponding to the claimed panoramic view) displayed on the monitor, and, therefore, the controlling 
portion 31 of computer 1 must control the processing algorithms for generating the graphics in the 
operation area 6A and the panorama operation area 6B; paragraph [0012], the user selects a desired 
point for the operation area and the system generates a photograph at the designated position selected; 
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Fig. 3, pointing device 14 is connected to computer 1 , indicating that computer 1 processes tine positional 
information from the user input); and encode the panoramic view data and the virtual view data for display 
(Koyanagi: paragraph [0043], captured images are displayed; paragraph [0051], pictures may be 

captured in a particular format). 

Koyanagi does not explicitly state that the virtual view data is generated from the buffered wide- 
angle image data. However, Kuban discloses an omniview motionless camera surveillance system, 
wherein the omnidirectional viewing system produces the equivalent of pan, tilt, zoom, and rotation within 
a selected field-of-view with no moving parts by direct mapping of the image region-of-interest into a 
corrected image using an orthogonal set of transformation algorithms (Kuban: column 2, line 51 -column 3, 
line 11). Since both Koyanagi and Kuban relate to generating a region-of-interest image from wide angle 
image data, one of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the motionless region-of-interest generation of Kuban with the system of Koyanagi in order to 
conserve power by reducing the need to move a pan-tilter device. 

Re claim 2, Koyanagi discloses a user input module configured to provide user command data to 
said processor (Koyanagi: Fig. 3, pointing device 14; paragraph [0045]); and said processor being further 
configured to determine the virtual view data based on the user command data (Koyanagi: paragraph 
[0044]). 

Re claim 3, Koyanagi discloses that the processor is further configured to determine reference 
data corresponding to an area in the panoramic view represented by the virtual view (Koyanagi: Fig. 1 ; 
paragraph [0043], "a frame 6C that represents the current position and the angle of view of the pan tilter 
and a pan tilter limiter 6D are superimposed to the panorama picture"). 

Re claim 4, Koyanagi discloses a first video camera system having a first video camera operably 
coupled to said buffer and said processor (Koyanagi: Fig. 3, lens block 15 and CCD 19 generate image 
data), said first video camera system operable to generate wide-angle image data (Koyanagi: Figs. 4A-4F 
and paragraphs [0021] and [0053] -[0054], image data is compiled into a panoramic view). 
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Re claim 11, Koyanagi discloses that the system transforms wide-angle image data received by 
the buffer into virtual view data corresponding to the virtual view and into panoramic view data 

corresponding to the panoramic view in real time (Koyanagi: Fig. 15; paragraph [0131], the flow of the 
control algorithm advances to relevant steps in accordance with a user input, indicating real-time 
processing). 

Re claim 12, Koyanagi discloses a display device operably coupled to said processor to display 
the panoramic view and the at least one virtual view (Koyanagi: Fig. 3, monitor 2). 

Re claim 14, Koyanagi discloses at least one reference window overlaid on at least one portion 
of the panoramic view, each overlaid portion defining the portion of the panoramic view to which the 
virtual view corresponds (Koyanagi: Figs. 1 and 2), and the at least one reference window having a size 
and a position determined according to the user command data (Koyanagi: paragraph [0043], user 

controls position of operation area/virtual view; Fig. 16B; Figs. IDA and 10B; paragraph [0094], zoom 
operation changes the size of the reference window within the panoramic view). 

Claim 16 recites the corresponding method for implementation within the system of claim 1 , and, 
therefore, has been analyzed and rejected with respect to claim 1 above. 

Re claim 17, Koyanagi discloses determining pan, tilt and zoom values (Koyanagi: Fig. 16A, step 
S13); and determining a portion of the buffered wide-angle data to transform into virtual view data for the 
virtual view based on the pan, tilt and zoom values (Koyanagi: paragraphs [0050]-[0051]). 

Re claim 18, Koyanagi discloses determining reference data based on the pan, tilt and zoom 

values (Koyanagi: paragraph [0043]). 

Re claim 20, Koyanagi discloses encoding reference data, virtual view data and panoramic view 
data for output (Koyanagi: paragraph [0051]). 

Re claim 21 , Koyanagi discloses a system for creating signals indicative of a graphical user 
interface from wide-angle image data corresponding to a monitored area, said system comprising: means 
for buffering wide-angle image data corresponding to the monitored area (Koyanagi: Fig. 3, storing portion 
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30); means for processing and generating, from said buffered wide-angle image data received from said 
storing means, panoramic view data of a panoramic view of the monitored area (Koyanagi: Fig. 3 and 
paragraph [0046], computer 1 comprises a controlling portion 31 , a video capture portion 29, and a 
storing portion 30; Figs. 1 and 2 and paragraph [0051], computer 1 processes graphics in the operation 
area 6A (corresponding to the claimed virtual view) and the panorama operation area 6B (corresponding 
to the claimed panoramic view) displayed on the monitor, and, therefore, the controlling portion 31 of 
computer 1 must control the processing algorithms for generating the graphics in the operation area 6A 
and the panorama operation area 6B; paragraphs [0053]-[0054], images merged to form panoramic 
pictures); and means for processing and generating virtual view data representing a virtual view of a 
portion of the panoramic view (paragraph [0012], the user selects a desired point for the operation area 
and the system generates a photograph at the designated position selected; Fig. 3, pointing device 14 is 
connected to computer 1 , indicating that computer 1 processes the positional information from the user 
input). 

Koyanagi does not explicitly state that the virtual view data is generated from the buffered wide- 
angle image data. However, Kuban discloses an omniview motionless camera surveillance system, 
wherein the omnidirectional viewing system produces the equivalent of pan, tilt, zoom, and rotation within 
a selected field-of-view with no moving parts by direct mapping of the image region-of-interest into a 
corrected image using an orthogonal set of transformation algorithms (Kuban: column 2, line 51 -column 3, 
line 11). Since both Koyanagi and Kuban relate to generating a region-of-interest image from wide angle 
image data, one of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the motionless region-of-interest generation of Kuban with the system of Koyanagi in order to 
conserve power by reducing the need to move a pan-tilter device. 

Re claim 23, Koyanagi discloses that the size and the position of each reference window 

determines pan, tilt and zoom values for the corresponding virtual view (Koyanagi: paragraph [0043], user 
controls position of operation area/virtual view; Fig. 16B; Figs. 10A and 108; paragraph [0094], zoom 
operation changes the size of the reference window within the panoramic view). 
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Re claim 26, Koyanagi discloses determining a position and a size of at least one reference 
window positioned over the portion of at least one of the at least one panoramic view corresponding to 
the virtual view, the position and size defined according to user command data (Koyanagi: paragraph 
[0043], user controls position of operation area/virtual view; Fig. 16B; Figs. 10A and 10B; paragraph 
[0094], zoom operation changes the size of the reference window within the panoramic view); and 
wherein the pan, tilt and zoom values are based upon the position and the size of the at least one 
reference window (Koyanagi: paragraph [0044], user input controls pan filter). 

6. Claims 9 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over Koyanagi 
et al. (US 20040257436 A1) in view of Kuban et al. (US 5359363 A) in view of lUlonroe (US 
20070182819 Al). 

Re claim 9, Koyanagi discloses a majority of the features of claim 9, as discussed above in claim 
4, and additionally a camera system operably coupled to the processor (Koyanagi: Fig. 3, camera portion 
11), said camera system having a camera and being configured to aim the camera at a portion of the 
monitored area according to pan, tilt and zoom command data (Koyanagi: paragraphs [0048]-[0049]), and 
configured to capture video image data (Koyanagi: Fig. 3, camera portion 1 1 outputs a video signal); and 
wherein the processor is further configured to communicate pan, tilt and zoom command data to cause 
the camera system to aim the camera at a portion of the monitored area (Koyanagi: paragraphs [0048]- 
[0049]), but Koyanagi does not specifically disclose that the camera system includes a second video 
camera; and wherein captured video image data from the second video camera is included in the virtual 
view. 

However, Monroe discloses a digital security multimedia sensor system, wherein panoramic 
views of monitored areas are created by merging images captured by a multitude of cameras (Monroe: 
Fig. 2, elements 10a-10h) and cameras within the array may be selectively scrutinized in order to track 
objects (Monroe: Fig. 16; paragraph [0106]). Since both Koyanagi and Monroe relate to panoramic image 
processing, one of ordinary skill in the art at the time of the invention would have found it obvious to 
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combine tine multiple cameras of Monroe with the photographing system of Koyanagi in order to permit 
transmission of the least amount of data to accomplish the desired image transmission (Monroe: 

paragraph [0024]). The combined system of Koyanagi and Monroe has all of the features of claim 9. 

Re claim 19, the combined system of Koyanagi and Poelstra discloses a majority of the features 
of claim 1 9, as discussed in claims 1 6-1 8 above. Koyanagi additionally discloses that the buffered wide- 
angle data is received from a first video camera system communicating pan, tilt and zoom commands to a 
camera system (Koyanagi: paragraph [0049]); and receiving virtual view data for the at least one virtual 
view (Koyanagi: paragraph [0012]) from the camera system (Koyanagi: paragraph [0012]), but Koyanagi 
does not specifically disclose a second virtual view and receiving virtual view data for the second virtual 
view, wherein captured video image data from the second video camera is included in the at least one 
virtual view. 

However, Monroe discloses a digital security multimedia sensor system, wherein panoramic 
views of monitored areas are created by merging images captured by a multitude of cameras (Monroe: 
Fig. 2, elements lOa-IOh) and cameras within the array may be selectively scrutinized in order to track 
objects (Monroe: Fig. 16; paragraph [0106]). Since both Koyanagi and Monroe relate to panoramic image 
processing, one of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the multiple cameras of Monroe with the photographing system of Koyanagi in order to permit 
transmission of the least amount of data to accomplish the desired image transmission (Monroe: 
paragraph [0024]). The combined system of Koyanagi and Monroe has all of the features of claim 1 9. 

7. Claims 5, 15, 24, and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Koyanagi et al. (US 20040257436 A1) in view of Kuban et al. (US 5359363 A) in view of Poelstra (US 
5563650 A). 

Re claim 5, Koyanagi discloses a majority of the features of claim 5, as discussed above in claim 
4, but Koyanagi does not specifically disclose that the first video camera system includes a fisheye lens. 
However, Poelstra discloses a device for producing and consulting panoramic images, wherein images 
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are produced using a fish eye lens and the images are transformed into panoramic images (Poelstra: 
column 1 , lines 42-48). Since both Koyanagi and Poelstra relate to producing panoramic images for 
review by a user, one of ordinary skill in the art at the time of the invention would have found it obvious to 
combine the fish eye lens of Poelstra with the photographing system of Koyanagi in order to produce a 
system capable of creating multiple panoramic views very quickly for consultation by the user (Poelstra: 
column 1 , lines 38-41 ). The combined system of Koyanagi and Poelstra has all of the features of claim 5. 

Re claim 15, Koyanagi disclose a majority of the features of claim 15, as discussed in claim 14 
above. Additionally, Koyanagi discloses that the panoramic view includes a first panoramic view, the first 
panoramic view corresponding to a first portion of the monitored area (Koyanagi: Figs. 4A-4E, a 
panoramic image representing a portion of the monitoring area is extracted); the virtual view includes a 
first virtual view, the first virtual view corresponding to a first portion of the first panoramic view (Koyanagi: 
Figs. 1 and 2, the operation area represents a selected portion of the panorama operation area), but 
Koyanagi does not specifically disclose that the at least one panoramic view includes a second panoramic 
view, the second panoramic view corresponding to the remaining portion of the monitored area, and a 
second virtual view, the second virtual view corresponding to a second portion of the second panoramic 
view; and a combination of the first panoramic view and the second panoramic view provide a 360° view 
of the monitored area relative to a vertical axis. 

However, Poelstra discloses a device for producing and consulting panoramic images, wherein 
images are produced using a fish eye lens and the images are transformed into panoramic images 
(Poelstra: column 1 , lines 42-48), and more than one panoramic image may be extracted from the initial 
fish eye image (Poelstra: Figs. 5 and 6; column 3, lines 42-46). Poelstra further discloses registering of 
complete surroundings in a single image with the help of a fish eye optic (Poelstra: column 1 , lines 50-51 ), 
wherein successive radial panoramic images are generated from the original fish eye image (Poelstra: 
column 1 , line 65, through column 2, line 7). By disclosing that the fish eye image registers the complete 
surroundings, Poelstra indicates that the fish eye image encompasses 360°, and, thus, one of ordinary 
skill in the art at the time of the invention would have found it obvious that the radial images generated 
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from the original fisin eye image would also encompass the complete surroundings. Poelstra also 
discloses that the fish eye image is converted to panoramic images with reference to the center of the fish 
eye image (Poelstra: column 2, lines 1 -2), wherein the center of the fish eye image corresponds to a 
reference axis, and an angle gamma indicates the maximum viewing angle with respect to an axis that is 
perpendicular to the axis corresponding to the center of the fish eye image (Poelstra: Fig. 7; column 3, 
lines 56-61). Since both Koyanagi and Poelstra relate to producing images for consultation by a user, 
one of ordinary skill in the art at the time of the invention would have found it obvious to combine the 
multiple panoramic image display of Poelstra with the panoramic/virtual view display of Koyanagi in order 
to produce a system capable of producing and consulting a large number of panoramic views quickly 
(Poelstra: column 1 , lines 38-41). The combined system of Koyanagi and Poelstra has all of the features 
of claim 1 5. 

Re claim 24, the combined system of Koyanagi and Poelstra discloses a majority of the features 
of claim 24, as discussed above in claim 15. Additionally, Koyanagi discloses that the at least one 
reference window is user-selectable for controlling the size and the position of the reference window to 
determine pan, tilt and zoom values for the corresponding virtual view (Koyanagi: paragraph [0043], user 
controls position of operation area/virtual view; Fig. 16B; Figs. 10A and 10B; paragraph [0094], zoom 
operation changes the size of the reference window within the panoramic view). 

Re claim 25, Koyanagi discloses a majority of the features of claim 25, as discussed above in 
claim 1 6. Koyanagi does not specifically disclose two panoramic views. However, Poelstra discloses a 
device for producing and consulting panoramic images, wherein the at least one panoramic view includes: 
a first panoramic view corresponding to a first portion of the monitored area (Poelstra: Fig. 6, transformed 
image 22; column 3, lines 3-4 and 42-46); a second panoramic view corresponding to a remaining portion 
of the monitored area (Poelstra: Fig. 6, transformed image 23; column 3, lines 3-4 and 42-46); and the 
first panoramic view and the second panoramic view combine to provide a 360° view of the monitored 
area relative to a vertical axis (Poelstra: column 1 , lines 50-51 , registering of complete surroundings in a 
single image with the help of a fish eye optic; column 1 , line 65, through column 2, line 7, successive 
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radial panoramic images are generated from the original fish eye image); and wherein said method further 
comprises encoding the first panoramic view, the second panoramic view, and virtual view of a portion of 
at least one of the first panoramic view and the second panoramic view for simultaneous display 
(Poelstra: Fig. 6; column 3, lines 3-4, two panoramic images are displayed simultaneously). Since both 
Koyanagi and Poelstra relate to producing images for consultation by a user, one of ordinary skill in the 
art at the time of the invention would have found it obvious to combine the multiple panoramic image 
display of Poelstra with the panoramic/virtual view display of Koyanagi in order to produce a system 
capable of producing and consulting a large number of panoramic views quickly (Poelstra: column 1 , lines 
38-41). The combined system of Koyanagi and Poelstra has all of the features of claim 25. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: 

a. Image reproduction apparatus with panoramic mode based on aspect ratio; Suzuki (US 
7206017 B1) 

b. Multi-Sensor Panoramic Network Camera; Kaplinsky (US 20050141607 A1) 

c. Visual user interface for use in controlling the interaction of a device with a spatial region; 

Lassiter (US 6624846 B1) 

d. System for a plurality of video cameras disposed on a common network; Metzger et al. 
(US 200601 36972 Al) 

e. Surround surveillance system for mobile body, and mobile body, car, and train using the 
same; Kumata et al. (US 20020005896 Al) 

f. Surround surveillance apparatus for mobile body; Kumata et al. (US 2002008001 7 Al ) 

g. Storing and processing partial images obtained from a panoramic image; Kotake et al. 
(US 71 03232 B2) 

h. Panoramic digital camera system; Whiting et al. (US 6034716 A) 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth 
in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date 
of this final action and the advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the mailing date of this final action. 
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be directed to CHRISTOPHER FINDLEY whose telephone number is (571)270-1 199. The examiner can 
normally be reached on Monday-Friday (8:30 AM-5:00 PM). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Christopher Kelley can be reached on 571-272-7331 . The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic Business Center (BBC) 
at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative 
or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272- 
1000. 
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